Skip to content

Enhance AI/BI dashboard skill with comprehensive widget documentation#374

Open
QuentinAmbard wants to merge 42 commits intomainfrom
dashboard-improvements
Open

Enhance AI/BI dashboard skill with comprehensive widget documentation#374
QuentinAmbard wants to merge 42 commits intomainfrom
dashboard-improvements

Conversation

@QuentinAmbard
Copy link
Copy Markdown
Collaborator

@QuentinAmbard QuentinAmbard commented Mar 25, 2026

Summary

Comprehensive improvements to the AI/BI dashboard skill documentation and tooling.

Documentation Improvements

  • Restructured skill files for better organization:

    • Split widget specs into basic (1-widget-specifications.md) and advanced (2-advanced-widget-specifications.md)
    • Renumbered filter docs to 3-filters.md, examples to 4-examples.md, troubleshooting to 5-troubleshooting.md
    • Condensed SKILL.md to focus on workflow and key guidelines
  • Added JSON skeleton section to SKILL.md showing required structure:

    • queryLines must be array of strings (not "query": "string")
    • Widgets must be inline in layout[].widget (not separate array)
    • pageType required on every page
  • Added Genie clarification: Genie is NOT a widget - link via genie_space_id param only

  • Moved Key Patterns to top of 4-examples.md for discoverability (page types, versions, formatting, filters, layout grid)

  • Clarified example is reference only - adapt to user's actual requirements, tables, and metrics

  • Added structural errors table to 5-troubleshooting.md with common JSON parse failures and fixes

  • Added critical widget version requirements table to SKILL.md - wrong version = broken widget

  • Replaced basic examples with comprehensive Sales Analytics dashboard (4-examples.md):

    • Complete dashboard with title, KPIs, area/pie/bar charts, and data table
    • Global filters (date range, region, department) binding multiple datasets
  • Added data validation guidance to verify dashboards tell the intended story before deploying

  • Added comprehensive date range filtering documentation with examples for relative date filters

  • Added TOP-N + "Other" bucketing guidance for handling high-cardinality dimensions in charts

MCP Tool Improvements

  • Added skill reading requirement to manage_dashboard docstring:

    • Must read 4-examples.md before creating dashboards
    • If unfamiliar, read full skill (SKILL.md), widget specs, then example
    • Emphasizes valid JSON is critical
  • Added genie_space_id parameter to create_or_update_dashboard for linking dashboards to Genie spaces

  • Added catalog and schema parameters to create_or_update_dashboard for setting defaults for unqualified table names

Test plan

  • Verify dashboard creation with new genie_space_id parameter
  • Verify dashboard creation with catalog/schema parameters
  • Review restructured documentation for clarity
  • Test Sales Analytics example JSON structure

🤖 Generated with Claude Code

Copy link
Copy Markdown
Collaborator

@calreynolds calreynolds left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can we separate this from the previous PR? I see both in here -- this one should just be ai/bi dashboards I think

Quentin Ambard and others added 7 commits March 28, 2026 11:40
Added missing documentation from production dashboard generation:

1-widget-specifications.md:
- Combo charts (bar + line on same widget) with version 1
- Counter number formatting (currency, percent, plain number)
- Widget name max length (60 characters)
- Color scale restrictions (no scheme/colorRamp/mappings)
- Quantitative color encoding for gradient effects
- Bar chart group vs stacked decision criteria with examples

2-filters.md:
- Date range picker complete example
- Multi-dataset filter binding (one query per dataset)
- Global filter performance note (auto WHERE clause)

SKILL.md:
- ORDER BY guidance for time series and rankings

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
When a dimension has too many values (50+ stores, products, etc.),
charts become unreadable. Added guidance to:

- Check cardinality via get_table_details before charting
- Use TOP-N + "Other" SQL pattern to bucket low-value items
- Aggregate to higher abstraction level as alternative
- Use table widgets for high-cardinality data

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
The codebase doesn't use async anywhere else, so remove the unused
async version of deploy_dashboard and keep only the synchronous one.
- Remove asyncio import
- Remove async deploy_dashboard function (was using asyncio.to_thread)
- Rename deploy_dashboard_sync to deploy_dashboard
- Update exports in __init__.py

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
Allow linking a Genie space to a dashboard by passing genie_space_id.
This enables the "Ask Genie" button on the dashboard UI.

The Genie space config is injected into the serialized_dashboard JSON
under uiSettings.genieSpace with isEnabled=true and enablementMode=ENABLED.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
Allow setting default catalog and schema for dashboard datasets via
the dataset_catalog and dataset_schema API parameters. These defaults
apply to unqualified table names in SQL queries.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
- Document field-based filtering (automatic IN_RANGE on date fields)
- Document parameter-based filtering (:date_range.min/max in SQL)
- Show how to combine both approaches in one filter
- Add guidance on when NOT to apply date filtering (MRR, all-time totals)
- Update SKILL.md tools table with new genie_space_id, catalog, schema params

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
- Split widget specs into basic (1-widget-specifications.md) and advanced
  (2-advanced-widget-specifications.md) files
- Add area chart, scatter plot, combo chart, and choropleth map documentation
- Rename files for consistent numbering (3-filters, 4-examples, 5-troubleshooting)
- Remove duplicate information across files (versions, naming rules, etc.)
- Add widget display formatting guidance (currency, percentage, displayName)
- Simplify SKILL.md quality checklist with link to version table
- Shorten verbose examples while preserving all critical information
- Clarify query naming convention for charts vs filters

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
@QuentinAmbard QuentinAmbard force-pushed the dashboard-improvements branch from 5586f10 to 618b2d4 Compare March 28, 2026 10:42
Quentin Ambard and others added 12 commits March 28, 2026 19:40
- Counter: full Pattern 2 example with CRITICAL field name matching note
- Table: disaggregated:true guidance and bold emphasis
- Line/Bar: x,y,color encodings and disaggregated guidance
- Pie: 3-8 category limit for readability
- 5-troubleshooting.md: Restore full troubleshooting content with version
  guidance, filter debugging, and detailed error explanations
- SKILL.md: Restore full 10-item quality checklist
- SKILL.md: Restore standard dashboard structure example
- SKILL.md: Restore cardinality guidance table (with softer 'suggested' language)
- Reduce docstring verbosity across all 18 tool files (~89% reduction)
- Keep all functional information while being concise
- Add skill references to complex tools (dashboards, vector search, genie, jobs, pipelines, lakebase, unity catalog, serving, apps, agent bricks)
- Maintain human readability with bullet points and structure
- Preserve critical warnings (ASK USER FIRST, CONFIRM WITH USER)
- Keep return format hints for AI parsing

Net reduction: 1,843 lines across 18 files

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
- agent_bricks.py: Add context for description, instructions, volume_path, examples
- genie.py: Add context for table_identifiers, description, sample_questions, serialized_space
- jobs.py: Add context for tasks, job_clusters, environments, schedule, git_source
- lakebase.py: Add context for source_branch, ttl_seconds, is_protected, autoscaling params,
  and sync source/target table names
- pipelines.py: Add context for root_path, workspace_file_paths, extra_settings, full_refresh

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
Tool consolidations:
- pipelines.py: 10→2 (manage_pipeline, manage_pipeline_run)
- volume_files.py: 6→1 (manage_volume_files)
- aibi_dashboards.py: 4→1 (manage_dashboard)
- vector_search.py: 8→4 (manage_vs_endpoint, manage_vs_index, query_vs_index, manage_vs_data)
- genie.py: 5→2 (manage_genie, ask_genie)
- serving.py: 3→1 (manage_serving_endpoint)
- apps.py: 3→1 (manage_app)
- file.py: 2→1 (manage_workspace_files)
- sql.py: 6→5 (manage_warehouse replaces list/get_best)
- lakebase.py: 8→4 (manage_lakebase_database, manage_lakebase_branch, manage_lakebase_sync, generate_lakebase_credential)

Key patterns:
- All consolidated tools use an action parameter
- Each action has required params documented in docstring
- Error messages specify which params are required
- Hot paths (query_vs_index, ask_genie) kept separate for clarity
- All skills updated with action tables and examples

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
Merged consolidated tool approach with dashboard-improvements params:
- Combined manage_dashboard actions with genie_space_id, catalog, schema params
- Updated SKILL.md to document consolidated tool with optional params

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
Test infrastructure:
- Add comprehensive integration tests for all MCP tools
- Add test runner script with parallel execution support
- Add fixtures for workspace, catalog, and resource cleanup
- Add test resources (PDFs, SQL files, app configs)

Bug fixes in databricks-tools-core:
- Fix workspace file upload for directories
- Fix job notebook path handling
- Fix vector search index operations
- Fix apps API responses
- Fix dashboard widget handling
- Fix agent bricks manager listing

Bug fixes in MCP server tools:
- Add quota skip handling for apps test
- Fix genie space operations
- Fix lakebase database operations
- Fix compute cluster lifecycle handling
- Fix dashboard operations

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
- Replace serialized_dashboard param with dashboard_file_path
- Tool reads JSON from local file for easier iterative development
- Update SKILL.md with new workflow documentation

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
- Change simple_dashboard_json fixture to simple_dashboard_file
- Update all manage_dashboard calls to use dashboard_file_path
- Add tempfile imports and tmp_path usage for update test

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
The Databricks SDK's w.apps.deploy() returns a Wait[AppDeployment] object,
not an AppDeployment directly. The previous code passed the Wait object
to _deployment_to_dict(), which caused getattr() to return None for all
attributes since the Wait object doesn't have them.

This fix uses wait_obj.response to get the actual AppDeployment object
before converting it to a dictionary.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
@QuentinAmbard
Copy link
Copy Markdown
Collaborator Author

Additional Fix: deploy_app SDK return type handling

Added commit a73db19 which fixes a bug in databricks_tools_core/apps/apps.py:

Problem

The deploy_app function was incorrectly handling the Databricks SDK's async operation pattern. The SDK's w.apps.deploy() returns a Wait[AppDeployment] object, not an AppDeployment directly. The previous code passed the Wait object to _deployment_to_dict(), which caused all attribute extractions via getattr() to return None.

Fix

Changed to use wait_obj.response to get the actual AppDeployment object before converting it to a dictionary.

-    deployment = w.apps.deploy(...)
-    return _deployment_to_dict(deployment)
+    # w.apps.deploy returns a Wait[AppDeployment], use .response to get the AppDeployment
+    wait_obj = w.apps.deploy(...)
+    return _deployment_to_dict(wait_obj.response)

Testing

  • Verified with integration test test_full_app_lifecycle - passed in ~5 minutes
  • The deployment details (deployment_id, source_code_path, state) are now correctly populated

The Databricks SDK's w.apps.deploy() returns a Wait[AppDeployment] object,
not an AppDeployment directly. The previous code passed the Wait object
to _deployment_to_dict(), which caused getattr() to return None for all
attributes since the Wait object doesn't have them.

This fix uses wait_obj.response to get the actual AppDeployment object
before converting it to a dictionary.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
@QuentinAmbard
Copy link
Copy Markdown
Collaborator Author

once #394 is merged we'll rebased and only the dashboard update will remain

Quentin Ambard and others added 5 commits April 1, 2026 09:23
- Add tools summary table at top of MCP Tools section
- Change code blocks from python syntax to plain text
- Add "# MCP Tool: <name>" comments to clarify these are tool calls, not Python code
- Move Supporting Tools table to main tools table

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
Remove garbage characters from widget documentation.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
- Use w.genie.trash_space() in _delete_genie_resource
- Add _find_space_by_name() using SDK's list_spaces with pagination
- Use w.genie.update_space() and w.genie.create_space() for space management
- Use w.genie.get_space() with include_serialized_space in _get_genie_space
- Fix validation to allow space_id for updates without display_name

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
- Add per-suite timeout in run_tests.py (10 min default, configurable)
- Improve apps test with better cleanup and assertions
- Add skip logic for quota-exceeded scenarios

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
…cumentation

Add detailed parameter documentation to all 9 Unity Catalog MCP tools:
- manage_uc_objects: Document parameters by object_type (catalog/schema/volume/function)
- manage_uc_grants: Add privilege lists per securable type
- manage_uc_storage: Detail credential and external_location parameters
- manage_uc_connections: Document connection_type options and create_foreign_catalog
- manage_uc_tags: Detail set_tags/unset_tags/query parameters
- manage_uc_security_policies: Document row filter and column mask parameters
- manage_uc_monitors: Detail monitor creation and refresh parameters
- manage_uc_sharing: Document share/recipient/provider resource types
- manage_metric_views: Detail dimension/measure format and query parameters

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
Quentin Ambard and others added 5 commits April 1, 2026 09:41
Add clear instructions requiring users to:
0. Review the databricks-aibi-dashboards skill for widget JSON structure
1. Call get_table_stats_and_schema() for table schemas
2. Call execute_sql() to test EVERY query before use

This prevents widgets from showing errors due to untested queries.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
- Add Design Best Practices section for default dashboard behaviors
- Change /tmp paths to ./ for less opinionated examples
- Update parent_path example to use {user_email} placeholder

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
- Replace basic NYC taxi examples with complete Sales Analytics dashboard
- Add critical widget version requirements table to SKILL.md
- Add data validation guidance to verify dashboards tell intended story
- Document key patterns: page types, KPI formatting, filter binding, layout grid

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
Require agent to read 4-examples.md before creating dashboards, and if
unfamiliar, read full skill documentation first. Valid JSON is critical.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
Quentin Ambard and others added 12 commits April 2, 2026 11:33
When a client cancels a long-running MCP request, there's a race condition
between the cancellation and normal response paths:

1. Client cancels request → RequestResponder.cancel() sends error response
   and sets _completed = True
2. Middleware catches CancelledError and returns a ToolResult
3. MCP SDK tries to call message.respond(response)
4. Crash: assert not self._completed fails

Fix: Re-raise CancelledError instead of returning a result, allowing the
MCP SDK's cancellation handler to properly manage the response lifecycle.

See: modelcontextprotocol/python-sdk#1153

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
When tools have an outputSchema (auto-generated from return type like
Dict[str, Any]), MCP SDK requires structured_content in all responses.
The middleware was returning ToolResult without structured_content for
error cases (timeout, exceptions), causing validation errors:

  "Output validation error: outputSchema defined but no structured output returned"

Fix: Include structured_content with the same error data in all error responses.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
- Migrate ka_create, ka_get, ka_sync_sources to use Python SDK
- Keep ka_update using raw API 2.1 due to SDK FieldMask bug (converts
  snake_case to camelCase but API expects snake_case)
- Fix find_by_name to sanitize names (spaces→underscores) before lookup
- Fix ka_create_or_update to lookup by name when no tile_id provided,
  preventing ALREADY_EXISTS errors on repeated calls
- Update MCP tool layer to use new flat response format
- Map SDK state values (ACTIVE, CREATING, FAILED) to endpoint_status
- Add integration test for updating existing KA via create_or_update

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
- Provide default description for knowledge sources when not specified
  (API requires non-empty knowledge_source.description)
- Move KA update test to after endpoint is ONLINE (update requires ACTIVE state)

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
…ations

FastMCP auto-generates outputSchema from return type annotations (e.g.,
-> Dict[str, Any]) but doesn't populate structured_content in ToolResult.
MCP SDK validation then fails: "outputSchema defined but no structured output"

Fix: Intercept successful results and populate structured_content from
JSON text content when missing. Only modifies results when:
1. structured_content is missing
2. There's exactly one TextContent item
3. The text is valid JSON that parses to a dict

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
…n crashes

The asyncio.to_thread() wrapper was only applied on Windows, but it's
needed on ALL platforms to enable proper cancellation handling.

Without this fix, when a sync tool runs longer than the client timeout:
1. Client sends cancellation
2. Sync tool blocks event loop, can't receive CancelledError
3. Tool eventually returns, but MCP SDK already responded to cancel
4. AssertionError: "Request already responded to" → server crashes

This was discovered when uploading 7,375 files triggered a timeout,
crashing the MCP server on macOS.

Extends the fix from PR #411 which added CancelledError handling in
middleware - that fix only works when cancellation can propagate,
which requires async execution via to_thread().

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
Setting structured_content causes MCP SDK to validate it against the
tool's outputSchema. For error responses, the error dict {"error": True, ...}
doesn't match the expected return type (e.g., Union[str, List[Dict]]),
causing "Output validation error: 'result' is a required property".

Fix: Only set structured_content for successful responses, not errors.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
- Add JSON skeleton section to SKILL.md showing required structure
- Add Genie note clarifying it's not a widget (use genie_space_id param)
- Move Key Patterns to top of 4-examples.md for discoverability
- Clarify example is reference only - adapt to user's actual requirements
- Add structural errors table to 5-troubleshooting.md

Root cause fixes:
- queryLines must be array, not "query": "string"
- Widgets must be inline in layout[].widget, not separate array
- pageType required on every page

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants